{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "# Multiclass sparse logisitic regression on newgroups20\n",
    "# 基于newgroups20数据集的多类稀疏Ligistic回归\n",
    "\n",
    "Comparison of multinomial logistic L1 vs one-versus-rest L1 logistic regression to classify documents from the newgroups20 dataset. Multinomial logistic regression yields more accurate results and is faster to train on the larger scale dataset.\n",
    "\n",
    "Here we use the l1 sparsity that trims the weights of not informative features to zero. This is good if the goal is to extract the strongly\n",
    "discriminative vocabulary of each class. If the goal is to get the best predictive accuracy, it is better to use the non sparsity-inducing l2 penalty instead.\n",
    "\n",
    "A more traditional (and possibly better) way to predict on a sparse subset of input features would be to use univariate feature selection followed by a traditional (l2-penalised) logistic regression model.\n",
    "\n",
    "基于newgroups20数据集，比较多项式与One-vs-Rest的L1惩罚的Logistic回归分类。多项式Logistic回归能够产生更精确的结果，并且在大规模数据集上训练更快。\n",
    "\n",
    "本例使用L1稀疏度，将非信息特征值的权重设置为零。这样做的好处是在每个分类文章下，可以更精准的提取特征词，通过这些词能更加精准地对文章分类。如果要获得最好的分类预测正确率，最好使用非稀疏的L2惩罚。\n",
    "\n",
    "对于输入特征是稀疏集的数据集进行预测，一种更为传统（可能更好）的方法是使用单变量特征选择，然后使用传统（L2惩罚）Logistic回归模型。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Automatically created module for IPython interactive environment\n"
     ]
    }
   ],
   "source": [
    "print(__doc__)\n",
    "# Author: Arthur Mensch\n",
    "\n",
    "import time\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "from sklearn.datasets import fetch_20newsgroups_vectorized\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "t0 = time.process_time()\n",
    "\n",
    "# We use SAGA solver\n",
    "# 设置求解方法为saga\n",
    "solver = 'saga'\n",
    "\n",
    "# Turn down for faster run time\n",
    "# 设置少量样本数，加快模型训练运行速度\n",
    "n_samples = 10000\n",
    "\n",
    "# Memorized fetch_rcv1 for faster access\n",
    "# 加载全部集合，包括训练集和测试集\n",
    "dataset = fetch_20newsgroups_vectorized('all')\n",
    "X = dataset.data\n",
    "y = dataset.target\n",
    "# 取前10000条数据\n",
    "X = X[:n_samples]\n",
    "y = y[:n_samples]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dataset 20newsgroup, train_samples=9000, n_features=130107, n_classes=20\n"
     ]
    }
   ],
   "source": [
    "# 对数据进行交叉验证，分为90%的训练集，10%的测试集\n",
    "# stratify = y: 依据标签y，按原数据集y中各类比例，分配给train和test，使得train和test中各类数据的比例与原数据集一样。 \n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y,\n",
    "                                                    random_state=42,\n",
    "                                                    stratify=y,\n",
    "                                                    test_size=0.1)\n",
    "# 得到训练集的样本数和特征数\n",
    "train_samples, n_features = X_train.shape\n",
    "# 得到标签的分类总数（一共有多少类）\n",
    "n_classes = np.unique(y).shape[0]\n",
    "\n",
    "print('Dataset 20newsgroup, train_samples=%i, n_features=%i, n_classes=%i'\n",
    "      % (train_samples, n_features, n_classes))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "20newsgroup数据集，训练数据集有9000个样本，有130107个特征值，20个类别"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[模型：One versus Rest, 求解方法：saga] 迭代次数: 1\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "E:\\DevSoft\\Python\\Python37\\lib\\site-packages\\sklearn\\linear_model\\sag.py:334: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
      "  \"the coef_ did not converge\", ConvergenceWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[模型：One versus Rest, 求解方法：saga] 迭代次数: 3\n",
      "ovr模型的测试准确率: 0.7410\n",
      "% ovr模型的非零系数, 每个分类的系数值:\n",
      " [0.27054655 0.66330021 0.80395367 0.73247404 0.67713497 0.73477984\n",
      " 0.40889422 0.48959702 1.01301237 0.56261385 0.60104376 0.332803\n",
      " 0.7094161  0.85083816 0.56876263 0.65715142 0.64408525 0.81163965\n",
      " 0.44271254 0.41120001]\n",
      "ovr模型迭代3次的运行时间(s): 3.62\n",
      "[模型：Multinomial, 求解方法：saga] 迭代次数: 1\n",
      "[模型：Multinomial, 求解方法：saga] 迭代次数: 3\n",
      "[模型：Multinomial, 求解方法：saga] 迭代次数: 7\n",
      "multinomial模型的测试准确率: 0.7450\n",
      "% multinomial模型的非零系数, 每个分类的系数值:\n",
      " [0.13296748 0.11759552 0.13296748 0.13988486 0.1268187  0.16140561\n",
      " 0.15218243 0.09069458 0.07762841 0.12143851 0.14910804 0.10837234\n",
      " 0.18830655 0.1245129  0.168323   0.21828188 0.11605832 0.07839701\n",
      " 0.06917383 0.15602543]\n",
      "multinomial模型迭代7次的运行时间(s): 4.99\n"
     ]
    }
   ],
   "source": [
    "# 设置模型参数\n",
    "models = {'ovr': {'name': 'One versus Rest', 'iters': [1, 3]},\n",
    "          'multinomial': {'name': 'Multinomial', 'iters': [1, 3, 7]}}\n",
    "\n",
    "for model in models:\n",
    "    # Add initial chance-level values for plotting purpose\n",
    "    # 为了绘图，设置初始参数\n",
    "    accuracies = [1 / n_classes]\n",
    "    times = [0]\n",
    "    densities = [1]\n",
    "    \n",
    "    # 获得模型参数（即得到name和iters）\n",
    "    model_params = models[model]\n",
    "\n",
    "    # Small number of epochs for fast runtime\n",
    "    # 用小的迭代次数进行迭代\n",
    "    for this_max_iter in model_params['iters']:\n",
    "        # 打印参数信息\n",
    "        # print('[model=%s, solver=%s] Number of epochs: %s' % (model_params['name'], solver, this_max_iter))\n",
    "        print('[模型：%s, 求解方法：%s] 迭代次数: %s' % (model_params['name'], solver, this_max_iter))\n",
    "        # 设置模型参数，初始化Logistic回归模型\n",
    "        lr = LogisticRegression(solver=solver,\n",
    "                                multi_class=model,\n",
    "                                C=1,\n",
    "                                penalty='l1',\n",
    "                                fit_intercept=True,\n",
    "                                max_iter=this_max_iter,\n",
    "                                random_state=42,\n",
    "                                )\n",
    "        t1 = time.process_time()\n",
    "        # 训练模型\n",
    "        lr.fit(X_train, y_train)\n",
    "        train_time = time.process_time() - t1\n",
    "        \n",
    "        # 交叉验证，预测测试集的分类\n",
    "        y_pred = lr.predict(X_test)\n",
    "        # 得到准确率\n",
    "        accuracy = np.sum(y_pred == y_test) / y_test.shape[0]\n",
    "        # 得到非0特征系数，并求每一行的平均值\n",
    "        density = np.mean(lr.coef_ != 0, axis=1) * 100\n",
    "        # 保存准确率，均值，训练时间\n",
    "        accuracies.append(accuracy)\n",
    "        densities.append(density)\n",
    "        times.append(train_time)\n",
    "    models[model]['times'] = times\n",
    "    models[model]['densities'] = densities\n",
    "    models[model]['accuracies'] = accuracies\n",
    "    # print('Test accuracy for model %s: %.4f' % (model, accuracies[-1]))\n",
    "    # print('%% non-zero coefficients for model %s, per class:\\n %s' % (model, densities[-1]))\n",
    "    # print('Run time (%i epochs) for model %s: %.2f' % (model_params['iters'][-1], model, times[-1]))\n",
    "    # 打印最后一次迭代的模型性能指标\n",
    "    print('%s模型的测试准确率: %.4f' % (model, accuracies[-1]))\n",
    "    print('%% %s模型的非零系数, 每个分类的系数值:\\n %s' % (model, densities[-1]))\n",
    "    print('%s模型迭代%i次的运行时间(s): %.2f' % (model, model_params['iters'][-1], times[-1]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "本例总运行时间 14.758 s\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEdCAYAAABZtfMGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzs3Xl4VOX1wPHvSUggJOwJiywmKCKrLAGtVFxQRFRAaxVq3RXtz622tdWqCBYVu7m0tlYLuFRFREVUqCtUrUsTIOwKCAhhDQECCdlzfn/cO8kkTJJJMpOZZM7nefJk7v7OEO6Zd7nnFVXFGGOMCTdRoS6AMcYY44sFKGOMMWHJApQxxpiwZAHKGGNMWLIAZYwxJixZgDLGGBOWLEA1IyKiInJiDdvXichZjVikOl+3tvdgmg8ReUZEHqjHcb1EJFdEooNRLhM+LECFARHZJiJFIpJYZX2Ge8NOrsc5nxeRmd7rVHWAqi5rUGHrIVTXrY6I9BCRl0UkW0TyROR/InJRqMtVGxFZJiIF7s15v4i8KSLdAnTeG2vYnuz+HbZo6LW8qeotqvo7P8q3TUTO9Tpuu6omqGppXa4nIteKyOfVbLtcRL4QkaMisqwu5zXBYwEqfGwFpngWRGQQEBe64jRPItIR+BwoAgYAicDjwCsiclkoy+an21Q1ATgRSAD+GOLyNBcHgCeAWaEuiKlgASp8vARc7bV8DfCi9w5Vv+lW941QRKYCVwK/dr9tv+OuL/8mKiLTRWS+iLwoIkfcZrhUr3P0c693yN02wWvb8yLyNxFZ4p7/vyLSVUSeEJGDIvKNiAz12t/7uiNF5Ev3vLtF5K8iElvbhyMik0Ukvcq6u0Rkkft6vIisd9/LThH5VTWnugvIBW5Q1T2qmq+qrwIPA38SEXHPpyJyi4hsct/T055t7vbrRWSDu+19ETm+mnKfJiJ7vJujROQSEVnt9Xmki8hhEdkrIn+u7bMAUNVDwEJgiNd5o0TkHhH5zq0dzncDMiLSSkT+5a4/JCJpItJFRB4GzgD+6v5b/tWf63tds6X7777L/XlCRFp6bf+1+++8S0RuFK8mXO9avogkisi7btkOiMhn7vt5CegFvOOW79dVa3Qi0lFE5rrXOCgiC+vyHtzP8yNVnQ/squuxJngsQIWPr4C2bmCIBq4A/lWfE6nqs8DLwO/dppCLq9l1AjAPaA8sAv4KICIxwDvAB0Bn4HbgZRHp63Xs5cD9ODWQQuBLYIW7vACo7kZbihMkEoEfAGOA//PjbS0C+opIH691PwFecV/PBm5W1TbAQOCTas5zHvCGqpZVWT8f50Z4kte6i4ARwCk47/d8ABGZBPwWuBRIAj4DXvV1MVX9CsgDzqmm3E8CT6pqW+AEtxy1EpFO7vU3e62+A5gEnAkcBxwEnna3XQO0A3oCnYBbgHxVvc8t/23u38pt/lzfy33AaTiB8hRgJM7fBSIyDvgFcC5Oje/MGs7zSyAT5/PsgvP5qqpeBWwHLnbL93sfx74EtMapEXfGqRGbZsACVHjx1KLOA74Bdgb5ep+r6mK3Lf8lnBsMODecBGCWqhap6ifAu3g1QQJvqepyVS0A3gIKVPVF91yvAUPxwT3mK1UtUdVtwD+o+cblOe4o8LanDG6gOhkncAEUA/1FpK2qHlTVFdWcKhHY7WP9bq/tHrNU9ZCqbgeWUlFbuRl4VFU3qGoJ8AgwpLpaFE7w8pS7DTCeioBWDJwoIomqmusGtJo8JSI5wH63rLd7bbsZuE9VM1W1EJgOXObWNIpxAtOJqlrq/jscruVa/rgSeEhV96lqFjADuMrddjkwV1XXuf9+M2o4TzHQDTheVYtV9TP1I1GoOH1wFwC3uP/uxar6nwa9IxM2LECFl5dwvl1fS5XmvSDZ4/X6KNDKvZkdB+yoUsv4HujutbzX63W+j+UEXxcUkZPcppw9InIY5+ae6GtfH16hIkj+BFjo3vgAfoRz4/9eRP4jIj+o5hz7cW6EVXXz2u5R9fPxvKfjgSfd5qhDOP0XAnQXkd+6TVG5IvKMV7kvdZu+LgVWqOr37rYbcGpt37jNbhdB+Qg3z3l+61WOO1S1HTAY6AD08Np2PPCWV7k24NRYu+D8bb0PzHObwn7v1pQb6jicvw2P7911nm07vLZ5v67qDzi1wQ9EZIuI3OPn9XsCB1T1oJ/7mybEAlQYcW9aW3FutG/62CUPpynDo2tNp2tAUXYBPUXE+++jF4Gp0f0dp3bYx23W+i3Ozd0fHwCJIjIEJ1B5mslQ1TRVnYjTxLOQ6pvKPgJ+VOW9gfNtfwew0Y9y7MBpTmzv9ROnql+o6iNuU1SCqt7ilm09zo37Aio376Gqm1R1ilvux4AFIhLvjnDznOeRqgVQ1TXATMC7b2wHcEGVcrVS1Z1uzWKGqvYHTsdpvvT0eTb0b8W75tiLin6c3VQOoD2rO4mqHlHVX6pqb+Bi4BciMsaP8u0AOopI+zqX3IQ9C1Dh5wbgHFXN87EtA+ebeGu3o/mGGs6zF+hdzzJ8jRMMfy0iMeI8w3QxTn9VQ7UBDgO5InIy8DN/D3Sb0xbgfNvuCHwIICKxInKliLRT1WL3/NUNQX4caAvMFmdgRysRmYLTl3K3P81KwDPAvSIywL1+OxH5cS3HvILTRzQaeN2zUkR+KiJJbm31kLva3+HTL+AENs8AlmeAhz1NjSKSJCIT3ddni8ggt3/zME6Tmuc6/v6ttHQ/L89PFE5T5f3utRKBaVT0nc4HrnP7VVu723wSkYtE5EQ32Hr+/Wotn6ruBpYAfxORDu7f6+ga3oNUeQ+t3JXR7usWQJS7LRA1TNMAFqDCjKp+p6rp1Wx+HGd49F6cm9PLNZxqNk6fzKG6jmpS1SKcm94FOE1efwOuVtVv6nKeavwKpxZxBHgOp7+qLl7B6XR/3Q1YHlcB29xmw1uAn/o6WFWzgR8CrYD1QDZOR/5VqupXWVT1LZzazjz3emtxPquavAqcBXyiqt7NiOOAdSKSizNgYrLbr+dPOYqApwDPw65P4vTJfSAiR3AG3pzqbuuKE9wP4zT9/YeKQPIkTl/VQRF5qoZL5uI033p+zsGpxaUDq4E1OANlZrrlW+KWbylO892X7nkKfZy7D07tNtfd729ez849ihMED4nv0ZlX4QTcb4B9wM9reA+nV3kP+W6z9lXu8t9xRjXm4/x9mhASm7DQGNMYRKQfTjBvWeXLhTE+WQ3KGBM04jzzFSsiHXBqne9YcDL+sgBljAmmm4Es4DucPiW/+xyNsSY+Y4wxYclqUMYYY8KSBShjjDFhyQKUaRBxEsHmi5Ok9ZA4Uxbc4uNB2OqOD8pUDvW5johcIyLLxUncmulmW2jhtb2jiLwlzhQd34vIT4JZZmMinQUoEwgXu0laj8eZruA3OM9hNTWtcZ6hScR5fmgMznNbHk/jPIfWBScH3d89D+s2JcH+MmBMoFiAMgGjqjmquggnE/s1IjIQQEQuFJGVbs1kh4hM9zrsU/f3ITfv3A9E5AQR+UScqSH2izO5YHkqGxH5jThTahwRkW89KXGkhukmfF3HR/n/7iYpLVLVnTgPQo9yzx2Pk+/vATep6+c4D8Ve5W6/VkQ+F5E/ug+8bhWR8od33WwTs8WZemKniMx0szrg1saGu69/6tb0+rvLN3oetJYapuYQkavd82SLyANy7NQqC8SZbuMwcK3UME2G+JjGRY6dJuMZEfnQ/Tf4j1RkrxAReVxE9olIjois9vwdGFNXFqBMwKnq/3CmTjjDXZWHk/etPXAh8DNxpqwAJ/UPQHs379yXOLn5HsVJNtoPJ4fbdABxpvy4DRjh1trOB7a556hpuglf16nNaGCd+/okoFRVvXP1rcKZ4sHjVOBbnBrY73HSKXny5L0AlOBMOzEUGAt45vb6D06WCc81t1CR4X20ux2qmZrDDWZ/w6nVdcOZVsM7sS/ARJxMEu1xAm+102T46Urgd+57zaAiq8lYt8wnude6AidbhzF1ZgHKBMsunHx5qOoyVV2jqmWquhon7U+1U2yo6mZV/VBVC90pHP7stX8p0BInjVOMqm5T1e/cbTVNN1EnInIdkErFjLUJQE6V3XJwcgt6fK+qz7lTjryAEyy6iEgXnFRIP1fVPFXdh5O2arJ73H+83t8ZOMHZs3wmFQGquqk5LsN5APZzN/3RNI5NsPqlqi50/w3yqXmaDH+8p6qfup/zfcAPRKSnW8Y2OFOhiDslia/pTYyplQUoEyzdcaahQEROFZGlIpIlzlxGt1DDFBsi0llE5rlNYYdxcsYlghO8cPqJpgP73P080zvUNN2E39za3SyczOCevHm5OElmvbXFySnoUT49h9c0IAluuWKA3V5l+wdOoldwAtAZItIViMbJTzhKRJJxakMZ7n4+p+agyrQW7rWr1lqqTnVR0zQZ/vC+Xi7Ov/Vx6swd9lecmuteEXlWRKp+bsb4xQKUCTgRGYEToDz9GK/g9Nf0dOcyeoaKKTZ8PSn+qLt+sNuc9VOv/VHVV1T1hzg3fsVJoQM1TDdRzXV8lX0cTpLQi90pLTw2Ai2k8oy+p1DRBFiTHTgJUhO9ytVWVQe472czznxTdwCfquoRnGA3FWdSyTJ3P59Tc1BlWgsRicOZnNBb1fdf0zQZlaZ1cQNnVT29tifg1JZ3ueV8SlWH4zR/ngTcXcNnY0y1LECZgBGRtu63+nnAv7xu8G1wJpUrEJGRONnMPbKAMipPp9AGp8ZySES643WDE5G+InKO26FfgJN12jMtQ7XTTVRznarlPwenL+VHbj9aOXf6kzeBh0QkXkRG4fTrvFTb5+I2cX0A/Mn9jKLEGQji3cz5H5y+NU9z3rIqyzVNzbEAuFhETheRWJzmutrm2KppmoxVwAARGSLOFBTTfRw/XkR+6F7vd8DXqrpDREa4NeYYnEBXgP/ThxhTiQUoEwjviDO9ww6c/og/A9d5bf8/nBv7EZwbYflkgm5z1MPAf93mr9NwbrDDcPp43qPy5I0tcZrf9uPUMjrjTHoINUw3Uc11qnoAp0ltsVTMZrukyvuIw5nS4VXgZ6rqTw0KnEEisThTfBzECSreM/v+Bycwf1rNMlQzNYdbhttxvhjsxml23IfvaS08apomYyPwEM70F5uoqAl7ewV4EKdpbzhOnxY4zZ7Pue/xe5ymxj/6ON6YWlkuPmOaGbfJ7RDOrMVbg3D+54FMVa3LqD9j6sxqUMY0AyJysTgzLcfj1FjWUDH83pgmyQKUMc3DRJxBCrtwZqed7Of09caELWviM8YYE5asBmWMMSYsWYAyxhgTlixAGWOMCUsWoIwxxoQlC1DGGGPCkgUoY4wxYckClDHGmLBkAcoYY0xYqvNEbqGWmJioycnJoS6GMcaYelq+fPl+VU2qbb8mF6CSk5NJT08PdTGMMcbUk4h8X/te1sRnjDEmTFmAMsYYE5YsQBljjAlLFqCMMcaEJQtQxhhjard6Pjw+EKa3d36vnh/0Sza5UXzGGAOwcOVO/vD+t+w6lM9x7eO4+/y+TBraPdTFap5Wz4d37oDifGc5Z4ezDDD48qBd1gKUMabJWbhyJ/e+uYb84lIAdh7K59431wBYkFKF0mIoKYCSwsq/SwuPXXfM7yL3t9e6tW9UBCeP4nz4+CELUMYYA5BXWMK27DxmvLOuPDh55BeX8of3vw19gCorq7jBlxbVEgz8CRiF1QQWX+dxl2ngTOkSBS3ioEVLaNEKio/63i8ns2HXqYUFKGNMWCkqKWP7gaNs3Z/Htv15bNmfx9b9uWzdn8few4U1Hrvr0NHqaw++buTV/a5TTaPK+rLihn8I0S3d4OAGiKq/W7aB+CSIjvW9vdLvmra18jqH97oqoeHxgU6zXlXtejT8vdbAApQxobJ6vtNEkpPp/EcfMy2ozSWNoqzMvbnXfCMvLc7n4OEjZB86zMHDhzl4OJfDubnk5eVRmH+UWIppSRHtKebMmFImxZTRNraMNl1LiY8qYe+BHKLLimhJES2lmJY4P7FSAr8ra9h7kKjab+itE73W+woItQWFGvaJjoWoMBu/NmZa5T4ogJg4Z30QWYAyJhSC0emsCmUlVWoIAWoyqramUXTssh+igUT3p6qSmBg02rlZR8e2Iiqm6g29DQWxHcjYlc9RbUFhWQxFxFAa1ZIf9u1O3x6JvgNHTUHBO7hEtQCR+v0bNFeev8lG/kJlAcqYUPhohu9O5/d+Absyqg8ctfVpaANrD4jzzbimG3rrTpVqCUUSQ05RNAeLhOyCKLLyYc9R2JOnHC6JolBjKCSGkqhYOrRtS2L7tiR1aEfXTu3ontiBnont6dS+LeIGohZ+1B66AV+v3Mmfq4zi6xvq/qfmbPDljV7DF9UGdqY1stTUVLVksabJKCuFg9tg33rYu975vW897N9Y/TGxbWr4pu9Hn0N0Lc1LNTVBVVN7KCgu5fvso2zdn+v0CWXlsS07j63789ifW1FrEoHu7eNISYynd2I8KYnxpCQlkNIpnuPat6JFdJg1XZmQEJHlqppa235WgzImEFThyJ6KALRvA+xdB1nfQomnpiTQIRm6DHD2LTx87Hna9YS71jZmycuVlJaRme0MTtjiDlDY6v7sysnH+7tsUpuWpCTGc26/LiS7gah3Yjw9O7amVUx0SMpvmh8LUMbUVf4hyPrGCUD7NlQEpfyDFfskdIHO/SD1eud3l/6QdDLExjvbq/ZBQaN0Oqsqew4XsDUrj63ZTk3IE4S2HzhKSVlFFGrTqgW9kxIYkdyBlMSepCQ5Qej4Tq1p0yomqOU0BixAGVO94gKnKc4TgPa6NaPDXs9+xLZxAlD/idC5f8VPfKeazx3kTueDeUXu8GxniPa2/UfLa0Xezw+1bBFFSmI8fbu2YdzAriR7Nc11jI9FbLCACaGgBigRGQc8iTNo55+qOqvK9seBs93F1kBnVW0fzDIZcwxPP1F5jcj9nf0dqHszj46FxL5w/OlObahzfycwtetZ/xFfDex0zissKa/9eJrjPEEpJ7/iWZzoKKFXx9akJMbzg96dymtCKYnxdG3biqgoC0ImPAUtQIlINPA0cB6QCaSJyCJVXe/ZR1Xv8tr/dmBosMpjTEU/kRuAPIMWqusn8q4VdToBohu/WauwpJQdB46ydf/R8odVt7gDFKo+tHpcu1YkJ8Zz0eBuTp9QUjzJnZx+oRgbnGCaoGDWoEYCm1V1C4CIzAMmAuur2X8K8GAQy2MiSf6hyv1DnkELBYcq9kno4gSf1OvdWlG/yv1EjaS0TNl1KL+8NuT9k3nwKF7dQnSMjyUlMZ4z+iQ5I+Tcn+RO8cTF2uAE07wEM0B1B7xzY2QCp/raUUSOB1KAT6rZPhWYCtCrV6/AltI0bcUFsP/bigDkCUqHd1bs07KtE3wGTILOA5zX/vQTBZCqkpVbWD482zNUe+v+PL4/cJSikornl+Jjo0lJimdwj3ZMGnIcKUnxpCQ6Q7XbtbbBCSZyBDNA+WrYru6hq8nAAlUt9bVRVZ8FngXnOajAFM80KWWlcGCrV43IHbRw4LuKh1PL+4lGefUT9XcGIDRSZ39OfvEx/UGe5dzCkvL9YqOj6NXJ6Rc65+TOTi3IHaCQ1KalDU4whuAGqEygp9dyD2BXNftOBm4NYllMU6EKR3ZXHjW3z/M8UYG7k0DHFCf4DLjEHcY9ADr2bpR+ooLiUuch1axjnxfKzqv80GqPDnGkJCYw/PgOJHdqTUpSAr0T4zmufRzRNjjBmBoFM0ClAX1EJAXYiROEflJ1JxHpC3QAvgxiWUw4Ku8nqjJooVI/UVcnAI24saJpLqlv0PuJikvLyDyY7w5MqBigsDUrj105BZX27dymJcmJ8ZzXv0ulfiF7aNWYhglagFLVEhG5DXgfZ5j5HFVdJyIPAemqusjddQowT5taziXjP08/kXeqn30bquknusQJQp4mutYdg1assjJl75GC8pqQd3Nc1YdW27ZqQUpSAqf27lSpOS45MZ6ElvY4oTHBYLn4TOCU9xOtqzxowVc/kWfUnGfQQpD6iVSVg0eLnRxyXvnjPK8LiisGJ7SKiSK5U8XwbM9Q7ZTEBDq0jrF+IWMCxHLxmeDx9BNVqhGtr7mfyFMj6njCsZOhBUBuYUmlviDvfHLeD622cB9aTU6MZ9SJieU55JLtoVVjwo4FqEhUl4ny8g9WDN0uH7Tgo5+oS3+3n8j7eaLWNRZj4cqd/KHKdAk1TdfteWh1S9axzwvtO3LsQ6spSfFcfEo3Z4h2YmtSEhPo0SHOHlo1pomwABVpqpsor7QIug6qXCvaux6OeA28bNnWq0bk9TxRPfqJFq7cyb1vrinPC7fzUD73vrmGsjJlRErHSqPjPFN+7zyYX+mh1U7uQ6ujT0qqmN4hKZ7jO9pDq8Y0B9YHFWkeH+gEpZpEt4SkkyonP+3SH9p2D1g/0ahZn7DzUH6t+3keWnVqQRXNcfbQqjFNl/VBGd9yMqvf9uPnnUELHXsHpZ/I264agtOsSwe5E93Fk5RgD60aE6ksQEWadj1816Da9XSa7hpJt/at2HWo4Jj13dvHMXmkpbMyxoD1FkeaMdOcab29NcJEeVUN79XhmHVxMdHcfX7fRi2HMSZ8WYCKNH3Hg0RDTGtAnJrTxU8FbKI8f2QePMpHG/Yx8Li2dG8fh+DUnB69dFCNo/iMMZHFmvgizdoFUFoI13wAvXwmlw+66YucGVf+cXUq3dvHhaQMxpjwZzWoSKIKabOdgRA9R4akCB+s28NHG/by83P7WHAyxtTIAlQk2bkC9qyGEdc32vQT3o4WlTDjnfX07dKG63+Y0ujXN8Y0LdbEF0nSZ0NMPAxqvP4mb09+vImdh/J5/ZYfWDYHY0yt7C4RKfIPwto3nMEQrdo2+uW/2XOY2Z9t5fLUHoxIDl6GcmNM82EBKlJkvOokck29vtEvXVam3P/WWtq0asE9F/Rr9OsbY5omC1CRQBXS50CPEdBtcKNffsHyTNK/P8i94/vRMT620a9vjGmaLEBFgm2fQfamkNSeDuQV8ciSDYxI7sBlw3o0+vWNMU2XBahIkD4HWrVv1FRGHrOWbCC3oISZkwbZXEvGmDoJaoASkXEi8q2IbBaRe6rZ53IRWS8i60TklWCWJyId2Qsb3oEhVzopjRpR2rYDzE/P5IYzUujbtU2jXtsY0/QFbZi5iEQDTwPnAZlAmogsUtX1Xvv0Ae4FRqnqQRHpHKzyRKyVL0FZCaRe16iXLS4t47631tC9fRx3junTqNc2xjQPwaxBjQQ2q+oWVS0C5gETq+xzE/C0qh4EUNV9QSxP5CkrheXPQ8poSGzcIDH7861s3JvL9AkDaB1rj9sZY+oumAGqO+A9r0Omu87bScBJIvJfEflKRMYFsTyRZ/NHztQaqTc06mUzDx7lyY82cV7/LpzXv0ujXtsY03wE86utrx7xqtP3tgD6AGcBPYDPRGSgqh6qdCKRqcBUgF69bK4gv6XPgYQucPKFjXpZTzLY6RMGNOp1jTHNSzBrUJlAT6/lHsAuH/u8rarFqroV+BYnYFWiqs+qaqqqpiYlJQWtwM3Koe2w8X0YdjVEN97U6JYM1hgTKMEMUGlAHxFJEZFYYDKwqMo+C4GzAUQkEafJb0sQyxQ5lr/gJIQddk2jXTKvsITpi9ZZMlhjTEAErYlPVUtE5DbgfSAamKOq60TkISBdVRe528aKyHqgFLhbVbODVaaIUVIEK16EPudD+5617x8gT328iV05BSyYMtSSwRpjGiyow6tUdTGwuMq6aV6vFfiF+2MC5dv3IG9fo2aO+GbPYWZ/vpUrUnuSaslgjTEBYF9zm6O02dC+F5w4plEuVzkZ7MmNck1jTPNnAaq5ydro5N4bfi1ERTfKJV9fvqM8GWwHSwZrjAkQC1DNzfK5EBUDQ69qlMsdyCvi0SXfMDK5oyWDNcYElAWo5qQ4HzJehn4XQ0LjZI16dLGbDPaSgZYM1hgTUBagmpO1b0JBDoxonMwR/9t6gNeXZ3LjGb05qYslgzXGBJYFqOYkfQ4k9oXjRwX9UkUlZdy/0EkGe8eYE4N+PWNM5LEA1VzsXgU7052h5RL8pjZPMtgZlgzWGBMkFqCai/Q50CIOTpkc9EvtOHCUJz/eyNj+XTjXksEaY4LEAlRzUHAYVr8Og34Ece2DfrkZ76xDEB60ZLDGmCCyANUcrH4NivMaJXOEkwx2H3edZ8lgjTHBZQGqqVN1mve6DYHuw4N6Ke9ksNeNsmSwxpjgsgDV1O34Gvatb5TakycZ7MOXDLRksMaYoLO7TFOXNhtatoVBlwX1Mt/sOcw/LRmsMaYRWYBqyvKyYf1CZ+RebHzQLlNWptz31lraWjJYY0wjsgDVlGX8C0qLgt689/ryHSz//iC/tWSwxphGVGuAEpHgj1s2dVdWBulzodfp0Llf0C5TKRnscEsGa4xpPP7UoJaLyKsiMjbopTH+27IUDm4Net4972Sw0ggZKowxxsOfANUHeBG4SUQ2ichDInJCkMtlapM+B1onOpnLg8SSwRpjQqnWAKWqZaq6RFV/DNwE3ABkiMjHIjKypmNFZJyIfCsim0XkHh/brxWRLBHJcH9urPc7iSSHd8G3S2DoT6FFy6BcwpLBGmNCrdYsn24f1JXA1cBB4C7gLWA48Brg84lNEYkGngbOAzKBNBFZpKrrq+z6mqreVu93EIlWvAha5syaGySeZLD/vDrVksEaY0LCnztPGvAKcLmqfu+1/isRea6G40YCm1V1C4CIzAMmAlUDlKmL0hJY/gKcOAY6BiebgyWDNcaEA3/6oPqq6oNVghMAqvpIDcd1B3Z4LWe666r6kYisFpEFItLT14lEZKqIpItIelZWlh9FbsY2/huO7Ara0HJVZfqidUSJJYM1xoSWPwFqsfdQcxHpICLv+XGcryFfWmX5HSBZVQcDHwEv+DqRqj6rqqmqmpqUlOTHpZux9NnQtjv0OT8op/9g/V4+/mYfPz/XksEaY0LLnwDVVVUPeRZU9SBwnB/HZQLeNaIewC7vHVQ1W1UL3cXncPq1THUObIHvPoFh10B04PuF8gpLmLFoHSd3tWSwxpjQ8yd57PY5AAAgAElEQVRAlYpI+ROaItLLz3OnAX1EJEVEYoHJwCLvHUSkm9fiBGCDn+eOTOlzQaJh2NVBOf2TbjLYmZMsGawxJvT8+Ro+DfiviHziLp8N/Ky2g1S1RERuA94HooE5qrpORB4C0lV1EXCHiEwASoADwLX1eA+RoaQQVv4LTh4PbbvVvn8dfbPnMLM/38rkEZYM1hgTHmoNUKr6nvu80w9w+pV+o6r7/Dm5qi4GFldZN83r9b3AvXUqcaRa/zbkH4DUwGeO8CSDbRcXw2/GWTJYY0x48LcdpwDYDuwFThSR04NXJONT+hzo2BtSzgz4qeenO8lg773gZEsGa4wJG/48qHs98EucIeJrgBHAV8BZQS2ZqbB3HWz/EsbOhKjA9g1l5xYy69/fMDLFksEaY8KLP3e7u4BUYJuqnoEz0m53UEtlKkufC9EtYciVAT/1o0u+cZLBTrJksMaY8OJPgCpQ1XwAEYlV1XWAdVQ0lsJcWDUPBlwCrQM7eOHrLdksWJ7JTaMtGawxJvz4M4pvt/ug7jvA+yJyAKcvyjSGtQug6EjAM0c4yWDXOslgz+kT0HMbY0wg+DOKb4L78gERGQO0A/zJJGEaShXSZkOXgdCzxsTxdfbPz7ewaV8us69JJS42OqDnNsaYQKgxQLkZyVeo6ikAqvpxo5TKOHaugD2r4cI/QQD7h3YcOMpTH29ibP8ujOlnyWCNMeGpxj4oVS0F1ouIrySvJtjSZ0NsAgy+ImCntGSwxpimwp8+qERgg4h8CeR5VqrqpUErlYH8g7D2DThlCrQM3AAGTzLY+8b3s2Swxpiw5k+AmhX0UphjZbwKJQUwInCZI/IKS5juJoO9dlRywM5rjDHB4M8gCet3amyqTuaIHiOg66CAnfbJjzexO6eAv/5kqCWDNcaEPX8ySRyhYh6nFjiJXwtVtW0wCxbRtn0G2Ztg0jMBO+WG3RXJYIcfb8lgjTHhz58aVHkHiIhEAZcCpwSzUBEvfQ60ag8DJgXkdE4y2DWWDNYY06TUqZ1HVctUdQFwXpDKY47shQ3vwNCfQkxgBjHMT9/Biu2H+O34fpYM1hjTZPjTxDfBazEKJy+fJW0LlpUvQVkJDL8uIKfLzi3k0SVOMtgfDbOnBYwxTYc/o/h+7PW6BNgGTAxKaSJdWSksf96ZUiPxxICc8tEl35BXWMLDlgzWGNPE+NMHdVVjFMQAmz+CnB3OtBoB4EkG+7OzTqCPJYM1xjQxtfZBichsN1msZ7mDiDznz8lFZJyIfCsim0Xknhr2u0xEVERS/St2M5U2GxK6wMkXNvhUnmSwPTpYMlhjTNPkzyCJYap6yLOgqgdx5oSqkZvH72ngAqA/MEVE+vvYrw1wB/C1v4Vulg5th00fwLCrITqmwafzJIOdMWGAJYM1xjRJ/gSoKBFp51kQkQ6AP3fQkcBmVd2iqkXAPHz3Xf0O+D3OtPKRa/nzTkLYYdc0+FSeZLDnD7BksMaYpsufAPUE8KWIPCgi04D/An/y47juwA6v5Ux3XTkRGQr0VNV3/Sxv81RSBCtegj7nQ/ueDTqVqvKgJxnsxZYM1hjTdPkzSGKuiCwHzsEZXn6Fqq7x49y+hoxp+Ubnod/HgWtrPZHIVGAqQK9evfy4dBPzzbuQty8geffeX7eXT9xksMdZMlhjTBPmz3NQI4ANqrraXW4jIqmqml7LoZmAd3WgB7DLa7kNMBBY5g5/7gosEpEJVc+tqs8CzwKkpqYqzU36HGjfC044p0GnySssYcY7lgzWGNM8+NPE9yxw1Gs5D/iHH8elAX1EJEVEYoHJwCLPRlXNUdVEVU1W1WTgK+CY4NTsZW10cu8Nvw6iGjaY4YmPNrI7p4CHLxloyWCNMU2eX4MkVLXMs+C+rnWQhKqWALcB7wMbgPmquk5EHqqSnSKyLZ8LUTEwtGGPm23YfZg5/93GlJGWDNYY0zz4k0liq4j8DKcmpcDPcLJJ1EpVFwOLq6ybVs2+Z/lzzmal6ChkvAz9J0BCUr1PY8lgjTHNkT81qJuBMcBe9+dM4KZgFipirHsLCnIg9foGneY1r2Sw7VtbMlhjTPPgzyi+vcBljVCWyJM+GxL7wvGj6n2K7NxCZi35hlMtGawxppnxZxRfS5yh4AOAVp71qjo1eMWKALsyYOdyGPeY84BuPT2y2EkGO9OSwRpjmhl/mvheBJKBi3DSEZ1ApGd9CIT0OdAiDk6ZXO9TfLUlmzdWZDJ1dG9LBmuMaXb8CVAnqeq9QK6qzgbG4Ty/ZOqrIAfWLIBBP4K49rXv74N3MtjbLRmsMaYZ8idAFbu/D4lIP5wHbI8PXpEiwOr5UJwHqfXPHPHcZ1vYvC+XhyZaMlhjTPPkzzDz2W6C2AdxnmlqDfgcKm78oOo073UbAt2H1esUOw4c5S+fOMlgzznZksEaY5onf0bxebJGLAWaYSK8Rrb9K9i3Hib8pV6HWzJYY0yksHw4jS19DrRsBwN/VK/DPclgf3HeSZYM1hjTrFmAakx5+2H9QmfkXmx83Q/3TgZ7enLgy2eMMWHEnynfj2kG9LXO+CHjZSgtgtTr6nV4RTLYQbSwZLDGmGbOn7vc//xcZ2pSVgbpc52sEZ371fnw9bu8k8F2CEIBjTEmvFRbExKRzkA3IE5EBlExAWFbnJF8pi62LIWDW+Gc++t8aFmZcv9CSwZrjIksNTXVXQhcjzPR4NNUBKgjwANBLlfzkz4HWidCv4vrfKgnGewff3yKJYM1xkSMagOUqs4F5orI5ao6vxHL1Pzk7IRvl8Dpt0OLlnU6dL8lgzXGRCh/+qA6i0hbABF5RkT+JyJjglyu5mXFi6BlMPzaOh/66OJvOFpUwsOXWDJYY0xk8SdATVXVwyIyFqe572fA74NbrGaktARWvAAnjoGOKXU61JMM9qYzenNiZ0sGa4yJLP4EKHV/XwDMVdXlfh5nADYugSO765x3z5LBGmMinT+BZpWILAYuBpaISAIVQatGIjJORL4Vkc0ico+P7beIyBoRyRCRz0Wkf92K3wSkz4G23aHP2DodZslgjTGRzp8AdR0wHRipqkdxJi2stTogItE4o/8uAPoDU3wEoFdUdZCqDsFpNvxzHcoe/rK/g+8+cfqeov1/tnnHgaM89fEmxg3oaslgjTERq9YApaqlQG+cvieAOH+OA0YCm1V1i6oWAfOAiVXOfdhrMR4/a2ZNxvLnQaJh6FV+H6KqTHt7LS2ihAcnNL8KpTHG+MufVEd/Bc4GfuquygOe8ePc3YEdXsuZ7rqq579VRL7DqUHdUU0ZpopIuoikZ2Vl+XHpMFBcACv/BSdfCG27+X3Y++v2sPTbLO467yS6tbNksMaYyOVPTeh0Vb0Zd5p3VT0A+PO0qK8x0cfUkFT1aVU9AfgN4DPNgqo+q6qpqpqalJTkx6XDwIZFkH8AUq/3+5DcwhJmvLOeft3aWjJYY0zE82tGXRGJwg0uItIJKPPjuEygp9dyD2BXDfvPAyb5cd6mIW02dDwBUs70+5AnPnSSwc6cNNCSwRpjIl61d0GvjOVPA28ASSIyA/gceMyPc6cBfUQkRURigcnAoirX8B4/fSGwqQ5lD19718GOr5ys5VH+BZr1uw4z94ttTBnZy5LBGmMMNefi+x8wTFVfFJHlwLk4zXY/VtW1tZ1YVUtE5DacaeKjgTmquk5EHgLSVXURcJuInAsUAweBaxr4fsJD+hyIbglDrvRr97Iy5b6Fa2gfF8NvxvUNcuGMMaZpqClAlfchqeo6YF1dT66qi4HFVdZN83p9Z13PGfYKc2HVazDgEmjd0a9D5qXtYOX2Q/zJksEaY0y5mgJUkoj8orqNqtq8nlkKlLULoOgIjPAvc8T+3EIe+7eTDPZSSwZrjDHlagpQ0UACvkfjGV9UncERXQZCjxF+HfLI4g2WDNYYY3yoKUDtVtWHGq0kzcHOFbBnNVz4Z/Aj2Hz5XTZvrtjJrWefYMlgjTGmipqGmNnX+bpKnw2xCTD48lp3dZLBrqFHhzhuO9uSwRpjTFU1BSib86ku8g/C2jec4NSy9trQc59t4busPH43caAlgzXGGB+qDVBuxgjjr4xXoaTAr8wR27MrksGefXLnRiicMcY0PZauIBBUnWefeoyEroNq2VV5cJElgzXGmNpYgAqEbZ9B9ia/ak+WDNYYY/xjASoQ0udAXAcYUHMqwdzCEqYvsmSwxhjjDwtQDXVkL2x4x0lrFFNzjeiJDzey90gBD19iyWCNMaY2dpdsqJUvQVkJDL+uxt3W7cph7hfbmDyiF8N6WTJYY4ypjQWohigrdWbNTTkTEk+sfrcy5f6Fay0ZrDHG1IEFqIbY/BHk7Kg1754nGexvx/ezZLDGGOMnC1ANkTYbErpC3/HV7rI/t5BZSzZYMlhjjKkjC1D1dWg7bPoAhl0N0THV7vbI4g3kF5daMlhjjKkjC1D1tfx5JyHs8OrnWPQkg506urclgzXGmDqyAFUfJUWw4iU4aRy06+FzF08y2J4dLRmsMcbUR03TbZjqfPMu5O2rMXOEJxns3GtHWDJYY4yph6DWoERknIh8KyKbReQeH9t/ISLrRWS1iHwsIscHszwBkz4H2veCE3wnfPckg71goCWDNcaY+gpagBKRaOBp4AKgPzBFRKpmR10JpKrqYGAB8PtglSdgsjY6ufeGXwdRx358qso0NxnstIstGawxxtRXMGtQI4HNqrpFVYuAecBE7x1UdamqHnUXvwJ8d+iEk+VzISoGhl7lc/O/1+5hmSWDNcaYBgtmgOoO7PBaznTXVecGYImvDSIyVUTSRSQ9KysrgEWso6KjkPEy9J8ACUnHbM4tLGHGO5YM1hhjAiGYAcrXQz/qc0eRnwKpwB98bVfVZ1U1VVVTk5KODQyNZt1bUJADqb4zRzxuyWCNMSZggjmKLxPo6bXcA9hVdScRORe4DzhTVQuDWJ6GS58NSSfD8acfs2ndrhye/2IbU0ZaMlhjjAmEYH7NTwP6iEiKiMQCk4FF3juIyFDgH8AEVd0XxLI03K4M2LncGVpeJSNEWZly31tuMtjzTw5RAY0xpnkJWoBS1RLgNuB9YAMwX1XXichDIjLB3e0PQALwuohkiMiiak4XeulzIKY1DL7imE2vpm0nY8ch7ruwH+1aV5/2yBhjjP+C+qCuqi4GFldZN83r9bnBvH7AFOTAmgUw8EcQ177Spv25hTy25BtO692RS4ZaMlhjjAkU68n3x+r5UJznM3PEI+85yWBnThpkyWCNMSaALEDVRtVp3jtuKHQfVmnTF9/t582VO7l59Amc2DkhRAU0xpjmyQJUbbZ/BfvWH1N7Kiwp5f6Fa51ksOdUP5uuMcaY+rFksbVJnwMt2zn9T16e+3QLW9xksK1iLBmsMcYEmtWgapK3H9YvhFMmQ2x8+ert2Uf5yyebLRmsMcYEkdWgapLxMpQWVWres2SwprkoLi4mMzOTgoKCUBfFNFOtWrWiR48exMTU7/EbC1DVKSuD9Llw/CjoXPHwrScZ7AMX9bdksKZJy8zMpE2bNiQnJ9sIVBNwqkp2djaZmZmkpKTU6xzWxFedLUvh4NZKtSdPMtj+3dpyzQ+axtRVxlSnoKCATp06WXAyQSEidOrUqUE1dKtBVSd9DrROhH4Xl6/yJIP9+0+HWTJY0yxYcDLB1NC/L7vL+pKzE75dAsOughYtAVi7M4e5/93KlJG9GGrJYI0xJugsQPmy4kXQMhh+LeAkg71/4Vo6tI61ZLAmYi1cuZNRsz4h5Z73GDXrExau3Nngc4oIV11VMflnSUkJSUlJXHTRRXU6T3JyMvv372/wPqrKzJkz6dOnDyeddBJnn30269atq1NZmopt27YRFxfHkCFD6N+/P1dffTXFxcX1OtcjjzwS4NI5LEBVVVoCK16AE8+FDsmAJYM1ZuHKndz75hp2HspHgZ2H8rn3zTUNDlLx8fGsXbuW/Px8AD788EO6dw9dTsunn36aL774glWrVrFx40buvfdeJkyYEPKRjiUlJUE57wknnEBGRgZr1qwhMzOT+fPn1+s8wQpQ1gdV1cYlcGQ3XPhnALKOWDJY0/zNeGcd63cdrnb7yu2HKCotq7Quv7iUXy9Yzav/2+7zmP7HteXBiwfUeu0LLriA9957j8suu4xXX32VKVOm8NlnnwFw4MABrr/+erZs2ULr1q159tlnGTx4MNnZ2UyZMoWsrCxGjhyJasVcqP/617946qmnKCoq4tRTT+Vvf/sb0dH+PUz/2GOPsWzZMlq3bg3A2LFjOf3003n55Ze54YYbSEhI4M477+Tdd98lLi6Ot99+my5dupCVlcUtt9zC9u3OZ/HEE08watSoSuc+9dRTmTNnDgMGOJ/JWWedxZ/+9CdOPvlkbr/9dtasWUNJSQnTp09n4sSJPP/887z33nsUFBSQl5fHyy+/zBVXXMHhw4cpKSnh73//O2eccQYJCQnk5uYCsGDBAt59912ef/55Xn/9dWbMmEF0dDTt2rXj008/rfZ9R0dHM3LkSHbudL5wlJaWcs8997Bs2TIKCwu59dZbufnmm9m9e/cxZXjvvffIz89nyJAhDBgwgJdfftmvz9ofVoOqKn0OtO0BJ50PwKOLLRmsMVWDU23r62Ly5MnMmzePgoICVq9ezamnnlq+7cEHH2To0KGsXr2aRx55hKuvvhqAGTNm8MMf/pCVK1cyYcKE8sCwYcMGXnvtNf773/+SkZFBdHS0zxvm+PHj2bWr8vyphw8fJi8vjxNOOKHS+tTU1PJmvry8PE477TRWrVrF6NGjee655wC48847ueuuu0hLS+ONN97gxhtv9Pk+PTWU3bt3s2vXLoYPH87DDz/MOeecQ1paGkuXLuXuu+8mLy8PgC+//JIXXniBTz75hFdeeYXzzz+fjIwMVq1axZAhQ2r8XB966CHef/99Vq1axaJFNc9kVFBQwNdff824ceMAmD17Nu3atSMtLY20tDSee+45tm7d6rMMs2bNIi4ujoyMjIAGJ7AaVGXZ38F3n8DZ90FUdHky2NvOPtGSwZpmrbaazqhZn7DzUP4x67u3j+O1m3/QoGsPHjyYbdu28eqrrzJ+/PhK2z7//HPeeOMNAM455xyys7PJycnh008/5c033wTgwgsvpEMHZ+DSxx9/zPLlyxkxYgQA+fn5dO58bLaXxYsXH7OuOqpa/uU0Nja2vH9s+PDhfPjhhwB89NFHrF+/vvyYw4cPc+TIEdq0aVO+7vLLL+e8885jxowZzJ8/nx//+McAfPDBByxatIg//vGPgBMsPAH3vPPOo2PHjgCMGDGC66+/nuLiYiZNmlRrgBo1ahTXXnstl19+OZdeeqnPfb777juGDBnCpk2buOyyyxg8eHB5mVavXs2CBQsAyMnJYdOmTXUuQ0NZDcrb8udBomHY1eXJYHt1bG3JYE3Eu/v8vsRVyTkZFxPN3ef3Dcj5J0yYwK9+9SumTJlSab13052HJ1j4atFQVa655hoyMjLIyMjg22+/Zfr06X6VoW3btsTHx7Nly5ZK61esWEH//k7WmJiYmPLrRkdHl/cNlZWV8eWXX5Zfd+fOnZWCE0D37t3p1KkTq1ev5rXXXmPy5MnlZX7jjTfKj92+fTv9+vUDnD46j9GjR/Ppp5/SvXt3rrrqKl588cVjPgfvvrJnnnmGmTNnsmPHDoYMGUJ2dvYx79nTB7V582a++uqr8pqWqvKXv/ylvExbt25l7Nix1ZYhWCxAeRQXwMp/wckXQpuu5clgZ0wcYMlgTcSbNLQ7j146iO7t4xCcmtOjlw5iUoD6Za+//nqmTZvGoEGDKq0fPXp0ebPRsmXLSExMpG3btpXWL1myhIMHDwIwZswYFixYwL59+wCnD+v777/3uxx33303d9xxR/mgjY8++ojPP/+cn/zkJzUeN3bsWP7617+WL2dkZPjcb/Lkyfz+978nJyen/L2ef/75/OUvfykPxitXrvR57Pfff0/nzp256aabuOGGG1ixYgUAXbp0YcOGDZSVlfHWW2+V7//dd99x6qmn8tBDD5GYmMiOHTuqLX+3bt2YNWsWjz76aHmZ/v73v5eP6tu4cSN5eXnVliEmJqbeIwBrEtQmPhEZBzwJRAP/VNVZVbaPBp4ABgOTVXVBMMtTow2LIP8AjLihPBns+EFdObuvJYM1BpwgFaiAVFWPHj248847j1k/ffp0rrvuOgYPHkzr1q154YUXAKdvasqUKQwbNowzzzyTXr16AdC/f39mzpzJ2LFjKSsrIyYmhqeffprjj6+c+WX8+PH885//5Ljjjqu0/vbbb+fgwYMMGjSI6Ohounbtyttvv01cXM1pzZ566iluvfVWBg8eTElJCaNHj+aZZ545Zr/LLruMO++8kwceeKB83QMPPMDPf/5zBg8ejKqSnJzMu+++e8yxy5Yt4w9/+AMxMTEkJCSU115mzZrFRRddRM+ePRk4cGD5gIm7776bTZs2oaqMGTOGU045pcb3MGnSJKZPn85nn33GjTfeyLZt2xg2bBiqSlJSEgsXLqy2DFOnTmXw4MEMGzYsoP1Q4qsKHZATi0QDG4HzgEwgDZiiquu99kkG2gK/Ahb5E6BSU1M1PT098AWefT7kZaG3pXHt88tJ33aAj395Fl3btQr8tYwJAxs2bChvSjImWHz9nYnIclVNre3YYDbxjQQ2q+oWVS0C5gETvXdQ1W2quhpo+FCghti7DnZ8BanXsWTdPv6zMYtfjO1rwckYY0IomAGqO+Dd6JnprqszEZkqIukikp6VlRWQwlWSPgeiW5Lb7wpmvLPOksEaY0wYCGaA8vXQUL3aE1X1WVVNVdXUpKSkBharisJcWPUaDLiEP3++n31HCnn4koGWDNYYY0IsmHfhTKCn13IPYFc1+4bOmteh6AjfJV/B819s5SeWDNYYY8JCMANUGtBHRFJEJBaYDNT8OHNjU4X0OWiXAfzii1g6xsfya0sGa4wxYSFoAUpVS4DbgPeBDcB8VV0nIg+JyAQAERkhIpnAj4F/iEjjpg3euRz2rOZ/nS5hVWaOJYM1xpgwEtSOFlVdrKonqeoJqvqwu26aqi5yX6epag9VjVfVTqpae2bJQEqfQ1lMPHeu78MPendi0hBLBmtMtVbPh8cHwvT2zu/V9ct87S3cptu49tprad26NUeOHClfd+eddyIifh3rSQ30xBNPcPTo0fJt48eP59ChQ7W9jQY5/fTTa90nIaFppWyL3JEARw/A2jf4Mn4M2cWx/G7SQEsGa0x1Vs+Hd+6AnB2AOr/fuaPBQSrcptsAOPHEE3n77bcBJ4XR0qVL61ymqgFq8eLFtG/fPqDlrOqLL74I6vlDIXID1Kp5UFLAzL0/4JYzT7BksCayLbkH5l5Y/c/bt0FxlWSxxfnO+uqOWXKPX5f2TLcBlE+34XHgwAEmTZrE4MGDOe2001i9ejUA2dnZjB07lqFDh3LzzTcfM93GyJEjGTJkCDfffDOlpaV1+iimTJnCa6+9BjjZG0aNGkWLFk7SnW3btjFw4MDyff/4xz8ek+vvqaeeYteuXZx99tmcffbZQEXtbdu2bfTr14+bbrqJAQMGMHbs2PLgnJGRwWmnncbgwYO55JJLytM3nXXWWdx1112MHj2afv36kZaWxqWXXkqfPn24//77y6/rqR3l5uYyZswYhg0bxqBBg8qDbVMUmQFKlbL0OayL6kteh37cerYlgzWmRqWFdVtfB+Ey3YZHnz59yMrK4uDBg7z66qvlSV39dccdd3DcccexdOlSli5desz2TZs2ceutt7Ju3Trat29fnq396quv5rHHHmP16tUMGjSIGTNmlB8TGxvLp59+yi233MLEiRN5+umnWbt2Lc8///wxSWBbtWrFW2+9xYoVK1i6dCm//OUvfSbdbQoic7qNbZ8Rlb2J2UW38NAVlgzWGC6YVfP2xwe6zXtVtOsJ173XoEuH43Qbl156KfPmzePrr7/mH//4R4PeX1UpKSnl01QMHz6cbdu2kZOTw6FDhzjzzDMBuOaaa8qn4wAn2zvAoEGDGDBgAN26dQOgd+/e7Nixg06dOpXvq6r89re/5dNPPyUqKoqdO3eyd+9eunbtGtD30RgiMkDl/fdZijSB0n4TOcuSwRpTuzHTnD4n72a+mDhnfQB4pttYtmxZpRpBfafb8GTlrq/JkyczbNgwrrnmGqKiKhqaWrRoQVlZRWa2+kwF37Jly/LX0dHR5U18/hwTFRVV6fioqKhjpoN/+eWXycrKYvny5cTExJCcnBzyKevrK6Ka+NIW/YN903vTetM7tKCES+N8p7U3xlQx+HK4+CmnxoQ4vy9+ylkfAOEy3YZHr169ePjhh/m///u/Suu7dOnCvn37yM7OprCw0GfWcYA2bdpUGglYm3bt2tGhQ4fyqe5feuml8tpUXeXk5NC5c2diYmJYunRpvd5/uIiYGlTaon8wcPn9xEkRCLShgJFrppMW24IRE24OdfGMCX+DLw9YQKoqXKbb8HbzzcfeF2JiYpg2bRqnnnoqKSkpnHyy7wf7p06dygUXXEC3bt189kP58sILL3DLLbdw9OhRevfuzdy5c/06rqorr7ySiy++mNTUVIYMGVJtGZuCoE23ESz1nW5jz/QT6cqxiWb3kETX6ZsDUTRjmhSbbsM0hnCdbiOsdFbfWdA7a80P3xljjAmNiAlQ+8R3FvR9ktjIJTHGGOOPiAlQO4bdTb7GVlqXr7HsGHZ3iEpkTOg1tSZ+07Q09O8rYgLUiAk3s3b4TPaQRJkKe0hi7fCZNkDCRKxWrVqRnZ1tQcoEhaqSnZ1Nq1b1n5k8YgZJGGMqKy4uJjMzs8k+I2PCX6tWrejRowcxMZVnifB3kETEDDM3xlQWExNDSkpKqIthTLUiponPGGNM02IByhhjTFiyAGWMMSYsNblBEiKSBTQ0uWk4EEAAAAYcSURBVFQiYE/oOuyzqMw+jwr2WVSwz6Kyhn4ex6uq74dTvTS5ABUIIpLuzwiSSGCfRWX2eVSwz6KCfRaVNdbnYU18xhhjwpIFKGOMMWEpUgPUs6EuQBixz6Iy+zwq2GdRwT6Lyhrl84jIPihjjDHhL1JrUMYYY8KcBShjjDFhKaIClIiME5FvRWSziNwT6vKEkojMEZF9IrI21GUJNRHpKSJLRWSDiKwTkWPnHo8gItJKRP4nIqvcz2NGqMsUaiISLSIrReTdUJcllERkm4isEZEMEQl61u6I6YMSkWhgI3AekAmkAVNUdX1ICxYiIjIayAVeVNWBoS5PKIlIN6Cbqq4QkTbAcmBSBP9tCBCvqrkiEgN8Dtypql+FuGghIyK/AFKBtqp6UajLEyoisg1IVW2cqcgjqQY1EtisqltUtQiYB0wMcZlCRlU/BQ6EuhzhQFV3q+oK9/URYAPQPbSlCh115LqLMe5PZHyT9UFEegAXAv8MdVkiTSQFqO7ADq/lTCL4JmR8E5FkYCjwdWhLElpuk1YGsA/4UFUj+fN4Avg1UBbqgoQBBT4QkeUiMjXYF4ukACU+1kXst0JzLBFJAN4Afq6qh0NdnlBS1VJVHQL0AEaKSEQ2A4vIRcA+VV0e6rKEiVGqOgy4ALjV7SoImkgKUJlAT6/lHsCuEJXFhBm3r+UN4GVVfTPU5QkXqnoIWAaMC3FRQmUUMMHte5kHnCMi/wptkUJHVXe5v/cBb+F0nQRNJAWoNKCPiKSISCwwGVgU4jKZMOAOCpgNbFDVP4e6PKEmIkki0t59HQecC3wT2lKFhqreq6o9VDUZ557xiar+NMTFCgkRiXcHESEi8cBYIKijgCMmQKlqCXAb8D5OJ/h8VV0X2lKFjoi8CnwJ9BWRTBG5IdRlCqFRwFU4344z3J/xoS5UCHUDlorIapwvdh+qakQPrzYAdAE+F5FVwP+A91T138G8YMQMMzfGGNO0REwNyhhjTNNiAcoYY0xYsgBljDEmLFmAMsYYE5YsQBljjAlLFqCMcYlIJ69h5ntEZKfXcqyf55grIn3rcM1zROQ0r+VbReTK+pTfj2u9JSLH17D9iWBnBjCmLmyYuTE+iMh0IFdV/1hlveD8vwlIXjYRmQnsV9UnAnG+Gq5zCnC/qv64hn1OAP6qqhcEsyzG+MtqUOb/27ufEJvCMI7j31/dMpQ/ixlFSBkWU2MGw2KYspE1omShJCk1bMQGjSyUIhtJxoLlbMwCpbGQfwtjqJlYMBbKAqnJxITGY3HeOx3TneYfM1fz+2zue7vPee85p25P77lvz2OjkFQtqUfSZaALWCTpiqTO1C/pZC72oaR6SQVJfZLOpr5KTyQtHDbvCmA/cDSt0holnZF0JDfXeUkPJL2U1JBWQa9TAi3Oszf1b3oh6ZKkUr/rPUB7ii9IupH6+vRIagaIiN50bVV/9w6aTYwTlNnY1ACtEbEmIt4DxyOiAagDtkiqKXHMfOB+RNSRVe3Yl/8wJYSrwLmIqI+IxyXmGIiIJrJSTDeBg0AtcEDSglTEdRvQmIq7FshK8gy3kazPFcA6oDIialMvsOu5uOdA46h3w2wKFKb7BMz+E70R8TT3fncqD1UAFpMlsOENDgci4k4aPwOaJvC9xXqR3UB3RHyAocZxS8jq5K0HOrOnj8zmz7YyRYuAT2n8hqzE1UXgNnA3F/cxXY/ZtHOCMhubr8WBpJXAYWBDRPSl6tYVJY75kRsPMrHf2/f0+is3Lr4vkLWRuRYRJ0aZZ6B4jhHxWdJqspYJzcAOoNjbpyLFmk07P+IzG795QD/wJbWL3zqJufqBuZM4vgPYJakShnYiLisR9wqoTjFVZBs92oBTwNpc3Cr+cYVqs7HyCsps/LrIHuf1AG+BR5OYqx1ok7QdODTegyOiW1IL0JE2R/wk+5/q3bDQW8Bmst5OS4HWtCMxgGMAkmYBy8n+hzKbdt5mbjYDSJoD3AM2RcTgCDE7gZqIaJnSkzMbgR/xmc0AEfENOE22WWIkAi5MzRmZjc4rKDMzK0teQZmZWVlygjIzs7LkBGVmZmXJCcrMzMqSE5SZmZWl33y6wEANJ+jLAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 初始化plt\n",
    "fig = plt.figure()\n",
    "ax = fig.add_subplot(111)\n",
    "\n",
    "# 针对两个模型分别绘制折线图\n",
    "for model in models:\n",
    "    name = models[model]['name']\n",
    "    times = models[model]['times']\n",
    "    accuracies = models[model]['accuracies']\n",
    "    # 绘制折线，横坐标为运行时间，纵坐标为模型准确率\n",
    "    ax.plot(times, accuracies, marker='o',\n",
    "            label='Model: %s' % name)\n",
    "    ax.set_xlabel('Train time (s)')\n",
    "    ax.set_ylabel('Test accuracy')\n",
    "ax.legend()\n",
    "fig.suptitle('Multinomial vs One-vs-Rest Logistic L1\\n'\n",
    "             'Dataset %s' % '20newsgroups')\n",
    "fig.tight_layout()\n",
    "fig.subplots_adjust(top=0.85)\n",
    "run_time = time.process_time() - t0\n",
    "# 打印总运行时间\n",
    "# print('Example run in %.3f s' % run_time)\n",
    "print('本例总运行时间 %.3f s' % run_time)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.2"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": false,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
